Skip to content

Conversation

jonathan-buttner
Copy link
Contributor

This PR allows the ai21 and llama integrations to show up in the services API for the inference API.

Reference issue: elastic/kibana#230463

@jonathan-buttner jonathan-buttner marked this pull request as ready for review August 22, 2025 16:56
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/ml-core (Team:ML)

KodeRad added a commit to elastic/kibana that referenced this pull request Aug 26, 2025
… labs and Llama Stack (#232098)

## Summary

Fixes #230463

The corresponding back-end [pull
request](elastic/elasticsearch#133233) has not
been merged yet to avoid showing AI21 Labs and Llama Stack providers
without icons in the list of available connector services. This PR is
intended to go first for that reason. Once we receive the updated list
of services, the front end will be ready to display them correctly.

I’ve been provided with the expected response objects:

### AI21
    {
        "service": "ai21",
        "name": "AI21",
        "task_types": [
            "completion",
            "chat_completion"
        ],
        "configurations": {
            "api_key": {
"description": "API Key for the provider you're connecting to.",
                "label": "API Key",
                "required": true,
                "sensitive": true,
                "updatable": true,
                "type": "str",
                "supported_task_types": [
                    "completion",
                    "chat_completion"
                ]
            },
            "rate_limit.requests_per_minute": {
"description": "Minimize the number of rate limit errors.",
                "label": "Rate Limit",
                "required": false,
                "sensitive": false,
                "updatable": false,
                "type": "int",
                "supported_task_types": [
                    "completion",
                    "chat_completion"
                ]
            },
            "model_id": {
"description": "Refer to the AI21 models documentation for the list of
available inference models.",
                "label": "Model",
                "required": true,
                "sensitive": false,
                "updatable": false,
                "type": "str",
                "supported_task_types": [
                    "completion",
                    "chat_completion"
                ]
            }
        }
    },
    
### Llama
```
{
        "service": "llama",
        "name": "Llama",
        "task_types": [
            "text_embedding",
            "completion",
            "chat_completion"
        ],
        "configurations": {
            "api_key": {
                "description": "API Key for the provider you're connecting to.",
                "label": "API Key",
                "required": true,
                "sensitive": true,
                "updatable": true,
                "type": "str",
                "supported_task_types": [
                    "text_embedding",
                    "completion",
                    "chat_completion"
                ]
            },
            "rate_limit.requests_per_minute": {
                "description": "Minimize the number of rate limit errors.",
                "label": "Rate Limit",
                "required": false,
                "sensitive": false,
                "updatable": false,
                "type": "int",
                "supported_task_types": [
                    "text_embedding",
                    "completion",
                    "chat_completion"
                ]
            },
            "model_id": {
                "description": "Refer to the Llama models documentation for the list of available models.",
                "label": "Model",
                "required": true,
                "sensitive": false,
                "updatable": false,
                "type": "str",
                "supported_task_types": [
                    "text_embedding",
                    "completion",
                    "chat_completion"
                ]
            },
            "url": {
                "description": "The URL endpoint to use for the requests.",
                "label": "URL",
                "required": true,
                "sensitive": false,
                "updatable": false,
                "type": "str",
                "supported_task_types": [
                    "text_embedding",
                    "completion",
                    "chat_completion"
                ]
            }
        }
    },
```

### Checklist

Check the PR satisfies following conditions. 

Reviewers should verify this PR satisfies this list as well.

- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This was checked for breaking HTTP API changes, and any breaking
changes have been approved by the breaking-change committee. The
`release_note:breaking` label should be applied in these situations.
- [ ] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [ ] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
- [ ] Review the [backport
guidelines](https://docs.google.com/document/d/1VyN5k91e5OVumlc0Gb9RPa3h1ewuPE705nRtioPiTvY/edit?usp=sharing)
and apply applicable `backport:*` labels.

### Identify risks

Does this PR introduce any risks? For example, consider risks like hard
to test bugs, performance regression, potential of data loss.

Describe the risk, its severity, and mitigation for each identified
risk. Invite stakeholders and evaluate how to proceed before merging.

- [ ] [See some risk
examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)
- [ ] ...

---------

Co-authored-by: Kibana Machine <[email protected]>
@jonathan-buttner jonathan-buttner merged commit 952f8b4 into elastic:main Aug 27, 2025
33 checks passed
@jonathan-buttner jonathan-buttner deleted the ml-expose-ai21-llama branch August 27, 2025 20:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

:ml Machine learning >non-issue Team:ML Meta label for the ML team v9.2.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants